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Abstract 



The Lefschetz fixed point tlieorem and its converse have many generahzations. 
One of these generahzations is to endomorphisms of a space relative to a fixed 
subspace. In this paper we define relative Lefschetz numbers and Reidemeister 
traces using traces in bicategories with shadows. We use the functoriality of this 
trace to identify different forms of these invariants and to prove a relative Lefschetz 
fixed point theorem and its converse. 

Introduction 

The goal of topological fixed point theory is to find invariants that detect if a 
given endomorphism of a space has any fixed points. The Lefschetz fixed point 
theorem identifies one such invariant. 

Theorem (Lefschetz fixed point theorem) . Let B be a closed smooth manifold and 

f-B^B 

be a continuous map. If f has no fixed points then the Lefschetz number of f is 
zero. 

The Lefschetz number is the alternating sum of the levelwise traces of the map 
induced by / on the rational homology of B. This is a relatively computable 
invariant. It gives a necessary, but not sufficient, condition for determining if a 
continuous map does not have any fixed points. 

If we put additional restrictions on the map /, such as requiring it to preserve 
a subspace of B, the Lefschetz number still gives a necessary condition for / to 
be fixed point free. However, this invariant ignores the relative structure and so 
is not the best possible invariant. For example, the identity map of the circle is 
homotopic to a map with no fixed points and so the Lefschetz number is zero. If 
this map is required to preserve a proper subinterval it is no longer homotopic to a 
map with no fixed points. 

There is a refinement of the Lefschetz number defined using the induced maps 
on the rational homology of the subspace and the relative rational homology. 

Theorem A (Relative Lefschetz fixed point theorem). Let A C B be closed smooth 
manifolds and 

f-B^B 

be a continuous map such that f{A) C A. If f has no fixed points then the relative 
Lefschetz number of f is zero. 
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The relative Lefschetz number of the identity map of the circle relative to a 
proper subinterval is not zero. 

Both of these theorems give a condition that implies that a continuous endomor- 
phism 

has a fixed point. In most cases they do not give a condition that implies / has 
no fixed points. To address this question a refined invariant and some restrictions 
on the spaces have to be introduced. This refined invariant is called the Nielsen 
number or the Reidemeister trace. 

Theorem (Converse to the Lefschetz fixed point theorem). Let B he a closed 
smooth manifold of dimension at least 3 and 

f-B^B 

he a continuous map. The map f is homotopic to a map with no fixed points if and 
only if the Reidemeister trace of f is zero. 

The Reidemeister trace is a partitioning of the Lefschetz number to reflect the 
ways fixed points can be changed by a homotopy of the original map. There is 
a generalization of the Reidemeister trace to a relative Reidemeister trace that is 
very similar to the generalization of the Lefschetz number to the relative Lefschetz 
number. 

Theorem B (Converse to the Relative Lefschetz fixed point theorem). Suppose 
A <Z B are closed smooth manifolds of dimension at least 3 and the codimension of 
A in B is at least 2. A map 

f-B^B 

such that f{A) d A is homotopic to a map with no fixed points through maps 
preserving the subset A if and only if the relative Reidemeister trace of f is zero. 

The goal of this paper is to provide definitions of the relative Lefschetz num- 
ber and relative Reidemeister trace and proofs of Theorems A and B that satisfy 
several requirements. First, the relative Reidemeister trace should detect if a map 
is relatively homotopic to a map with no fixed points. It is not necessary for the 
relative Reidemeister trace to provide a lower bound for the number of fixed point 
of a given map. Second, the invariants should be trace-like. This means that they 
can be described using the duality and trace in bicategories defined in [26]. The 
relative Reidemeister trace should to be compatible with the approach of [HI [18] . 
Those papers give a proof of the converse to the Lefschetz fixed point theorem that 
is different from the standard simplicial proof. Finally, the relative Reidemeister 
trace should be compatible with an equivariant generalization of the Reidemeister 
trace described in [25] . 

While the Lefschetz number and the Reidemeister trace have long established 
definitions, the relative forms of these invariants are less settled. Versions of the 
relative Lefschetz number have been defined in [U [15] and of the relative Reide- 
meister trace in [Ml [551 IMl [33 13S] ■ The papers [551 1311 [3S] are primarily interested 
in determining lower bounds for the number of fixed points and so are generaliza- 
tions of the Nielsen number. The invariants defined in [24l [36] are more trace-like, 
but the definition are still motivated by connections to the Nielsen number. None 
of these invariants satisfy all of our conditions above, and none of them exactly 
coincide with the definitions given here. 
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In this paper we give proofs of Theorems A and B foUowing the outline of [26] . We 
use duahty and trace in bicategories with shadows to define two forms of the relative 
Lefschetz number and the relative Reidemeister trace. Then, using functoriality, 
we show different invariants coincide. Finally, we generalize Klein and Williams's 
proof of the converse to the Lefschetz fixed point theorem in [19] to complete the 
proof of the converse to the relative Lefschetz fixed point theorem. 

In the first two sections we will recall the necessary definitions of duality and trace 
in symmetric monoidal categories and in bicategories with shadows. In [Section "3l 
we will describe some examples of bicategories with shadows and generalize some 
results from 26J that describe specific examples of duals. 

In [Section 41 we apply this category theory to the relative Lefschetz number. In 
Sections [5] and [6] we define the relative Reidemeister trace. We describe how to 
compare these invariants to each other and how to compare them to the relative 
Nielsen number defined by Schirmer and Zhao. In Sections [7] and [5] we give a proof 
of the converse to the relative Lefschetz fixed point theorem following the proofs 
given by Klein and Williams in 19, 18J. In [Section 91 we include some formal results 
omitted from the third section. 

We assume the reader is familiar with the basic definitions of Nielsen theory. 
References for this topic include [31 [TCj . 

Acknowledgements. I would like to thank Peter May, Gun Sunyeekhan, and 
Bruce Williams for many helpful conversations and comments on previous drafts 
of this paper. I would also like to thank Xuezhi Zhao for sending me one of his 
preprints. 

Preliminaries. We fix some conventions and recall a fact about cofibrations. 

Definition 0.1. Let A C B and X C Y . A map /: i? — > y is a relative map if 
fiA) c X. 

We will write this 

/: {B,A)^iY,X). 

A homotopy H : B x I ^ Y is a relative homotopy if H\axi factors through the 
inclusion X C Y. 

Definition 0.2. [34l 3.1] A relative map /: {B,A) {B,A) is taut if there is a 
neighborhood N{A) of A in B such that f {N{A)) C A. 

We will use this condition to isolate the fixed points of A from those of B\ A. 

Lemma 0.3. [34, 3.2] If A C B is a cofibration then any relative map f: {B, A) — > 

(_B, A) is relatively homotopic to a taut map. 

We will assume A C B is a cofibration and all relative maps 

/: iB,A)^iB,A). 

are taut. If a relative map is not taut it is implicitly replaced by a relatively 
homotopic map that is taut. Since all invariants defined here are invariants of the 
relative homotopy class, the choice of replacement does not matter. 
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1. Duality and trace in symmetric monoidal categories 

Duality and trace in symmetric monoidal categories is a generalization of the 
trace in linear algebra that retains many important properties. The trace in a 
symmetric monoidal category satisfies a generalization of invariance of basis and 
has nice functorial properties. The Lefschetz fixed point theorem is one application 
of the functoriality properties of the trace. This is a very brief summary of [9]. For 
more details see [9], [211 Chapter III], or [2T. 

Let be a symmetric monoidal category with monoidal product (8), unit S, and 
symmetry isomorphism 

X (g)Y (g) X. 

Definition 1.1. An object ^ in "if is dualizahle with dual B if there are maps 

ly. S ^ A®B 

and 

e: B®A^S 

such that the composites 

A'^ S® A s- A® B® A ^ A® S A 

and 

B = B®S ^B®A®B-^S®B':^ B 

are the identity maps of A and B respectively. 

The most familiar example of a symmetric monoidal category is the category of 
modules over a commutative ring R. The tensor product is the monoidal product. 
If M is a finitely generated projective i?-module, M is dualizable and the dual of 
M is Homfl(M, R). The evaluation map 

e : Homi^ (M, R) ®r M R 

is defined by e((j),m) = (pirn). Since M is finitely generated and projective the 
dual basis theorem implies there is a 'basis' {mi, m2, . . . , to„} with dual 'basis' 
{m'l, m'2, . . . , m^}. The coevaluation is given by linearly extending 

77(1) = ^ mi ® m'i. 

The category of chain complexes of modules over a commutative ring R is also 
symmetric monoidal. The dualizable objects are the chain complexes that are 
projective in each degree and finitely generated. 

Definition 1.2. If A is dualizable and f : A A is a.n endomorphism in the 
trace of /, tr(/), is the composite 

S A® B A® B B ® A—^ S ■ 

The trace of an endomorphism in the symmetric monoidal category of vector 
spaces over a field is the sum of the diagonal elements in a matrix representation. 
The trace of an endomorphism in the category of chain complexes of modules over 
a commutative ring R is called the Lefschetz number. 
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Proposition 1.3. Let F : 'i^ —>■ S> be a symmetric monoidal functor, A be an object 
of ^€ with dual B, and 

F{A)(x)F{B) F{A®B) 

and 

S& ^ F{S^^) 

be isomorphisms. Then F{A) is dualizahle with dual F{B). If f : A A is an 
endomorphism in ^, F{ti-{f}) = tr{F{f)). 

The stable homotopy category is a symmetric monoidal category. There is also 
a way to express duality for spaces without using spectra. 

Definition 1.4. A based space X is n-dualizable if there is a based space Y and 
continuous maps 77 : S" X AY and e : Y A X ^ such that the diagrams 



?jAid 

S" AX ^ X AY AX 



id Ae 



Y AS 



id A'q 




(crAl)7 



5"' AF 



X AS'' 

commute up to stable homotopy. 

The map a: S"' 5" is defined by cr{v) = —v. 

Proposition 1.5. [HI III.4.1, III.5.1] 23, 18.6.5] 

(1) If M is a closed smooth manifold that embeds in M™, then M4. is dualizahle 
with dual Tv, the Thom space of the normal bundle of the embedding of AI 
in R™. 

(2) If L is a closed submanifold of a closed smooth manifold M that embeds in 
R™, then M U CL is dualizahle with dual Tvm U CTv^. 

(3) If B is a compact ENR that embeds in M", i?+ is dualizahle with dual the 
cone on the inclusion R" \ B R" . 

(4) If B is a compact ENR that embeds in R" and A is a sub ENR of B , then 
BUCA is dualizahle with dual (R" \A)U C(R" \ B). 

Here C denotes the cone. If A C -B then B U CA is the mapping cone on the 
inclusion A ^ B. The base point of i? U CA is the cone point. 

The trace of an endomorphism of spaces regarded as a map in the stable ho- 
motopy category is called the fixed point index. The index is the stable homotopy 
class of a map 

S" S" 

and so is an element of the 0*'' stable homotopy group of S*", ttq. For other descrip- 
tions of the fixed point index see [31 |E] . 

The index of the identity map of a space X is called the Euler characteristic of 
X and it is denoted x(^). This is consistent with the classical definition of the 
Euler characteristic. 

Since the rational homology functor is strong symmetric monoidal, [Proposition l3| 
implies that the fixed point index of a map / is equal to the Lefschetz number of 
H^{f). Since the fixed point index of a map with no fixed points is zero, this implies 
the Lefschetz fixed point theorem. 
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2. Duality and trace for bicategories with shadows 

Unfortunately, the Reidemeister trace cannot be defined as a trace in a symmetric 
monoidal category, ft can be defined using the more general trace in a bicategory 
with shadows. Duality and trace in a bicategory are very similar to duality and trace 
in a symmetric monoidal category but are more flexible. This is a brief summary 
of the relevant sections of [25 and [2^. For more details see [231 Chapter 16], [26] . 
or [22. 

Definition 2.1. [20l f .0] A bicategory ^ consists of 
(f ) A collection ob^. 

(2) Categories ^{A, B) for each A,B ^ oh3§. 

(3) Functors 

: ^{B, C) X ^(A, B) ^{A, C) 
Ua-*-> -^{A, A) 

for A, B and C in ob^. 
Here * denotes the category with one object and one morphism. The functors 
are required to satisfy unit and associativity conditions up to natural isomorphism 
2-cells. 

The elements of ob.^ are called 0-cells. The objects of £§{A, B) are called 1-cells. 
The morphisms of ^{A, B) are called 2-cells. 

The most familiar example of a bicategory is the bicategory Mod with 0-cells 
rings, 1-cells bimodules, and 2-cells homomorphisms. The bicategory composition 
is tensor product. 

Definition 2.2. [23l 16.4.1] A 1-cell X e Sg{B,A) is right dualizable with dual 
Y G ^{A, B) if there are 2-cells 

such that 




Y X 



commute. 

The map rj is called the coevaluation and e is called the evaluation. 

If i? is a (not necessarily commutative) ring and M is a finitely generated 
projective right i?-module then M is a right dualizable 1-cell in Mod with dual 
Horn/? (M;i?). The evaluation map 

e : RomRiM, R) ®z M ^ R 

is defined by e{4>,m) = (j>{m). This is a map of i?-i?-bimodules. Since M is finitely 
generated and projective there are elements {mi,TO2, . . . ,rnn} and dual elements 
{m[, 7712, ... , ^n} of Homi{(M, R) so that the coevaluation map 

7^:Z^M®R Homfl(Af, R) 
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is defined by linearly extending 77(1) ~ "^rrii ®m[. This is a map of abelian groups. 

Unlike the symmetric monoidal case, we need more structure before we can define 
trace. The additional structure is a shadow. 

Definition 2.3. [551 4.4.1] A shadow for ^ is a functor 

to a category ^ and a natural isomorphism 

e:iXQY))^iYQX)) 
for every pair of 1-cells X e £§{A, B) and Y e SS{B, A) such that 

{{X or)© z))^^iz (X © Y))) ^{{Z © X) © Y)) 



(X (y © Z)))^^i{Y Z) © X} HY (Z © X)} 

(Z © Ua)) (Ua Z} {Z Ua} 

commute. 

Let P be an i?-i?-bimodule. Let N[P) be the subgroup of P generated by 
elements of the form 

rp — pr 

for p e P and r e R. Then the shadow of P is P/N{P). 

Definition 2.4. [26l 4.5.1] Let X be a dualizable 1-cell and/: Q Q X ^ X Q P 
be a 2-cell in The trace of / is the composite 

M = {Q c/a)) ^ ((Q © X © Y)) 

/©id 

((X © p © y)) ((p © y © X)) ((p © Ub)) = (P)). 

If Af is a finitely generated projective right P-module and f : M ^ M is a map 
of right i?- modules the trace of / is the trace defined by Stallings in 30] . 

A functor of bicategories P is a shadow functor if there is a natural transforma- 
tion 

V':((PH))^P((H) 
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such that 

{FX FF))— ^((FF F{X))) 



(FiXGY)} 
FiiXQY})- 



(FiYQX))) 
FgY QXj) 



commutes for all 1-cells X and Y where X QY and Y Q X are both defined. 
Proposition 2.5. 26, 4.5.7] Let X be a right dualizable 1-cell in with dual Y, 

f: QQX^XQP 

he a 2- cell in ^ and F : d§ -> d§' be a shadow functor. IJF{X)QF{Y) F{XGY), 
F{X) F{P) — > F{X P), and Up(B) ^ F{Ub) are isomorphisms and f is the 
composite 



ther 



FQ FX F{Q X) F{X P) 



{{FQj^iFP} 



■FXQFP 



FiQf^FiP)) 



commutes. 



We will use this proposition to compare different forms of the Lefschetz number 
and Reidemeister trace. 



3. Some examples of bicategories with shadows 

The classical descriptions of fixed point invariants require a choice of base point. 
When working with a single space this isn't a problem. With fiberwise spaces, equi- 
variant spaces, or pairs of spaces, choosing base points requires addition conditions 
on the space. We can avoid these choices by using groupoids rather than groups. 

In this section we describe a generalization of the bicategory Mod that we will use 
to defined fixed point invariants without choosing base points. In this bicategory we 
replace rings by categories, modules by functors, and homomorphisms by natural 
transformations. 

Let Y he a, symmetric monoidal category with monoidal product ® and unit S. 

Definition 3.1. A category jz/ is enriched in if for each a,6 G ob(^), £^{a,b) 
is an object of Y and the composition for jz/, 

c) i/(a, 6) £/{a, c), 

is a morphism in 'V. 
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For pairs of enriched categories and define an enriched category ^ (g)^ with 
objects pairs (a, 6) where a € oh^/ and b E ohSS. If a, a! E oh^/ and b, b' £ oh^, 
then 

{£/ ® i^)((a, b), {a\ 6')) - a')) ® (-^^l^ &'))■ 

Definition 3.2. An enriched distributor is a functor ^ (g)^°P ^ 1^ such that 
the action of morphisms of £/ and ^ on ^ are maps in Y. 

This type of functor is also caUed an ^-^-bimodule. li F: £/ — + ^ is an 
enriched functor and W :^ ® SS°'^ Y is a. distributor define a new distributor 

^-f^ : ^ (g) i^°P ^ r by W^{a, b) = ^{F{a), b). 

Definition 3.3. An enriched natural transformation 77: j?^ — > is a natural 
transformation where the maps 

rja.y. 3r{a,b) ^^ia,b) 

are maps in f for all a E oh^/ and b G ob^. 

Enriched categories are the 0-cells of a bicategory we denote by (of. The 1-cells 
are the distributors enriched in Y. The 2-cells are enriched natural transformations. 
If JT: ^ (g) ^ r and ^: ^ (g "^"P ^ r are two distributors, ^ ^ is a 
distributor (g> "^"^ ^ f. For a e oh{£/) and c e obC^), J:" ^(a, c) is the 
coequalizer of the actions of ^ on ^ and 

Y[ ^(a,6)0i^(6',6)0^(fe',c)^^ Y[ ^(a,6) ® ^(&,c) 

b,b' eoh.-g 6eob.« 

If ^ y is an enriched functor, the shadow of 3f, ((^)), is the 

coequalizer of the two actions of on 3f, 

Y[ ^(a, a') ^(a, a') ]J 2f{a,a) 

a,a'£oh{.e/) aGob(j3') 

In Chapter 9] we observed that if ^ is a connected groupoid, a distributor 

is dualizable if and only if 2^ (a) is dualizable over si {a, a) for any a G ob^/. The 
categories we will use here and in |25| to define relative and equivariant fixed point 
invariants are not usually groupoids, but we can extend the results from |26j to 
describe these examples. 

Definition 3.4. [22l II. 9. 2] A category is an El-category if all cndomorphisms 
are isomorphisms. 

In an El-category there is a partial order on the set of objects: a; < y if 7^ 

0. 

Let Chjj be the symmetric monoidal category of chain complexes of modules 
over a commutative ring R and chain maps. Let s/ he a category enriched in the 
category of modules over R. This can be regarded as a category enriched in chain 
complexes concentrated in degree zero. 

Definition 3.5. A functor J^T: jz/ — > Ch/j is supported on isomorphisms if Sy{f) 
is the zero map if / is not an isomorphism. 



10 



KATE PONTO 



If 3^' is supported on isomorphisms it only 'sees' a disjoint collection of groupoids 
rather than the entire category si . 

Let B{£/) be a choice of representative for each isomorphism class of objects in 

Lemma 3.6. If -S^ : si°^ —> Ch^ and?V : £/ CUji are supported on isomorphisms 
then 

c6B(i/) 

The proof of this lemma is in lSection 91 The idea of the proof is to use lDefinition 3.5l 
to show that 

satisfies the universal property that defines ^ 

Lemma 3.7. Let 5C and W satisfy the conditions of lLemma 3.6\ If S^(c) is 
dualizable as an £/(c, c) -module with dual ?V(c) for each c £ B{£/) then is 
dualizable with dual W . 

The idea of this proof is to use ILemma 3.61 and the coevaluation and evaluation 
maps for each X {c) to define coevaluation and evaluation maps for 3^ . This proof 
can also be found in lSection 91 

Another choice for is the symmetric monoidal category of pointed topological 
spaces, Top^. The bicategory (ojop. has 0-cells categories enriched in based spaces 
and 1-cells distributors enriched in based spaces. The 2-cells in (ojop. are natural 
transformations enriched in Top^. 

If : ^ Top^ and : si ^ Top^ are enriched functors ^ is the bar 
resolution ^^). This is the geometric realization of the simplicial space 

with n-simplices 

]J ^(ao) A ^(ai, ao)+ . . . A ^(a„, a„_i)+ A <3/ (a„). 

The definition of the shadow is similar. If : si si°^ — > Top^ is a enriched 
functor, the shadow of iF, ((.2°)), is the cyclic bar resolution C(^, JF). This is the 
geometric realization of the simplicial space with n-simplices 

11 ^(o„,ao) Ai2/(ai,ao)+... A^(a 

ao ,ai ,. . . ,aTi €^obi/ 

Let ^ be a category enriched in based spaces. Let [/^ : si S2i°^ — > Top^ be 
defined by Uj^/{a,a') = si(a\a). Composition in si defines the action of si and 
si°P on U^. 

Definition 3.8. An enriched functor ^ : si Top^ is n-dualizable if there is a 
functor ii : si°P — > Top^, a map ry: 5*" B{^,si, ^V), and an j2/-£/-equi variant 
map e: ii A — > S"" A Uj^ such that the usual triangle diagrams commute up to 
^-equivariant homotopy. 

We will use the ideas of ILemma 3.71 to produced dual pairs in this bicategory, 
but we will not prove a general characterization. 
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Definition 3.9. If JT : ^ Top^ is dualizable, 0^: ® £/°p Top^ is an 
enriched functor and / : — ^ <^ is an enriched natural transformation the 
trace of / is the stable homotopy class of the composite 

5" -^{Sr ^))-^((^ 0^0 ^))^^((^ 0^0 ^))^^ 5" A((^)) . 

4. The relative Lefschetz number 

The global and geometric relative Lefschetz numbers can both be described us- 
ing a classical approach, but we will describe them using duality and trace in a 
bicategory. The formal structure gives a different perspective on these invariants 
and is a starting point for the more complicated invariants we will consider in the 
later sections. 

Definition 4.1. The relative component category Hq{B^A) of a pair {B^A) has 
objects the points of B. The morphisms of no(i?, A) are 

\ix e B \ Aand [x] = [y] G tio{B) 
if X e B \ Aand [x] / [y] £ 7ro(B) 
iix.y e Aand[x] = [y] G ■ko{A) 
ifx,?/ e Aand[x] ^ [y] G 7ro(A) 
\i X ^ A, y ^ A 

The composition is defined by the rules 

^o* = * 0O0 = 

0O* = *O0 = 0. 

For most pairs of spaces A C B this category is an El-category but not a 
groupoid. For example, if x G B \ A, y G A, and [x] — [y] G tto{B) then 
Ilo{B,A){x,y) — * and Uo{B,A){y,x) — 0. The relative component category 
is similar to the equivariant component category in [321 1.10.3]. 

If A and B are connected and B \ A is nonempty this category has two isomor- 
phism classes of objects. 

If a; G A, let A{x) be the component of A that contains x. If y G B, let B{y) be 
the component of B that contains y. 



no{B,A){x,y) = { 



B\A{x) 



Definition 4.2. The relative component space, B\A, of the pair {B,A) is the func- 
tor 

no(B,^)°p -.Top, 

defined by 

A{x)+ if a; G A 

Blx)/{AnB{x)) ifx^A 

The morphisms A{x) A{x) and B{x)/{A n B{x)) B{x)/{A n B{x)) are the 
identity maps. The map A{x) B{x)/{A D B{x)) is the inclusion of A{x) as the 
base point. 

Recall that C(A n is the cone on A n The base point is the cone 

point. Since ^ C B is assumed to be a cofibration B{x) yjC{A(^B{x)) is homotopy 
equivalent to B{x)/{A f^B{x)). 

Lemma 4.3. If A and B are both compact ENR's or closed smooth manifolds then 
B\A is dualizable. 
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Remark 4.4. Starting with the proof of this theorem we will focus on the case of 
closed smooth manifolds. The results in this section and Sections [5] and [6] have 
versions for compact ENR's as well. The statements and proofs for compact ENR's 
are very similar to those for closed smooth manifolds. 

Some of the results in [Section 71 have only been shown for manifolds. 



Proof. Define a functor D{B\A): Uo{B,A) Top, by 

D(BlA)(x)-[ ^(^(^) + ) ^^^^^ 
^[^\^)yx) I D{B{x)LlCiAnB{x))) \ix<^A 

where D(v4(x) + ) and D{B(x) U C{A n Bix))) denote the duals of and 
B(x) U C{A n B{x)) with respect to an embedding of B in M" as described in 
[Proposition 1.5[ The morphisms are the identity maps or the inclusion. 

To simplify notation, consider the case where A and B are both connected. The 
general case is similar. The evaluation for this dual pair is a natural transformation 



e: DiB\A) Q B\A ^ S'' A {UoiB,A))+. 

Let X £ A and y £ B\A represent the isomorphism classes of objects of Ilo{B, A). 
Then e consists of four maps 

D{BUC{A)) A (B/A) S*" 

D{A+) A (B/A) 5" 

D{BUC{A)) AA+ * 

D{A+)aA+ S" 

By naturality, the second map must be the constant map to a point. Since A^ and 
BU C (A) are both dualizable the evaluations for these dual pairs are the first and 
fourth maps. 

Note that B{B\A, Uo{B, A), D{B\A)) is equivalent to 

iA+ A D{A+)) V [{B/A) A D{B U CA)]. 
The dualizability of A+ and B U C{A) provide coevaluation maps 

Va: S^^A+AD{A+) 

Vb/a ■■ S"' -> B/A A D{B U C{A)). 
The coevaluation for this dual pair is the composite 

5" ^ S"" V S*" 

{A+ A D{A+)) V [{B/A) A D{B U CA+)]. 

Verifying that these maps describe a dual pair can be checked on x and y sepa- 
rately. The conditions reduce to conditions checked for [Proposition iTS] □ 

Let n^(i?, A) be the functor 

no(B,A) xno(B,A)°P^Top, 

defined by Hg (_B, y) = Tli3{B, A){f{y),x)+. The left action is composition. 
The right action is given by applying / and then the composition. 
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A relative map / : {B,A) {B,A) induces a natural transformation 
7:B\A^B\AQnl{B,A). 
Since SjA is dualizable, the trace of / is defined. 

Definition 4.5. The relative geometric Lefschetz number of /, iB\Aif)i is the trace 
of/. 

The relative geometric Lefschetz number is the stable homotopy class of a map 

and so it is an element of the 0*'' stable homotopy group of ((IIq A)}. This group 
is denoted ^^{([{[{B, A)}). It is the free abelian group on the set((n^'(B,^)}}. bmce 
the relative geometric Lefschetz number is defined to be the trace of / it is an 
invariant of the relative homotopy class of /. 

If Ho (A) is the set of component of A, (IIq {A)} is 

{[x]eMA)\[f\A{x)]^[x]}. 

If X is a closed smooth manifold, f : X X is a continuous map and F is an 
isolated subset of the set of fixed points of / let i{F, f) be the sum of the fixed 
point indices of the fixed points in F. See [8J for the definition of the fixed point 
index of an isolated set of fixed points. 

Lemma 4.6. There is an isomorphism 

iuliB,A)}-iuliA)}U{{uliB)} 

and the image of iB\A{.f) under this isomorphism is 

^ z(Fix(/)nA(a;),/)[x]+ ^ z(Fix(/) n \ A), /))[y]. 

^4^iiA))) ye((n^(s))) 

Since / is taut i(Fix(/) n AJ) = i(Fix(/) n A, /U) and 

z(Fix(/) n (B \ A), f) = j(Fix(/), /) - z(Fix(/) n A, fU). 

The first equality follows from commutativity of the index and the definition of a 
taut map. See [35l 3.5] for a proof of the second equality. 

Proof. Assume A and B are connected and A is a proper subset of B. These 
assumptions restrict the number of components of ((nQ(_B, v4))). The proof is similar 
for the general case. 

The set ((IIq (S, A))j is defined to be the coequalizer 

Y[^ y Ho (S, A) (x, y)+ A (S, A) (y, x) =^ U. (B, A) {x, x) ((HjJ {B, A))) . 

Since Ilo{B,A){x,y) is empty if x & A and y ^ A, this coequahzer splits as two 
coequalizers. One is over pairs {x,y) where x,y ^ A and the other is over pairs 
{x, y) where x,y ^ A. Each of these coequalizers consists of a single element. 

This isomorphism is compatible with lLemma 4.31 so the image of iB\A{f) under 
the projection to the first summand is the trace of / restricted to A. As observed 
after |Proposition 1.5| this is the fixed point index of f\A- 

The image of iB\Aif) under the projection to the second summand is the trace 
of f/A: B/A B/A. The fixed points of f/A are the fixed points of f\B\A and 
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the point that represents A. The point that represents A is the base point and so 
its index does not contribute to the trace of f/A, see [HI III. 8. 5]. □ 

The second component of iB\A{f) is the index defined in |T51 1.1]. 

Example 4.7. Let J be a nonempty, proper, connected subinterval of S-^. Let 
/: (S'\ J) {S^, J) be the identity map. Then isi\.j{f) = (1, -1). 

Corollary 4.8. ///: {B,A) {B,A) has no fixed points then ig^j<^{f) = 0. 

Proof. Since / has no fixed points j(Fix(/),/) = 0. To compute the relative geo- 
metric Lefschetz number of / we replace / by a relatively homotopic map g that 
is taut. The map g can be chosen so that /|^ — yj^. Then i{Fbc(g),g) = and 
z(Fix(.g) n A, gU) = i(Fix(/) n A, fU) - 0. 

Since / has no fixed points and f\A = g\A, i(Pix{g) Ci A,g) ~ 0. Since g is taut 

z(Fix(g) n {B \ A), g) = z(Fix(g), g) - z(Fix(g) n A, gU) = 0. 

□ 

Let Zno(-B, A) be the category with the same objects as no(i3, A). For objects 
X and y of no(-B, A) 

Zno(S,A)(x,2/) 

is the free abelian group on no(i3, A)(a;, y). Composing B\A with the rational 
homology functor defines a functor 

H,(B\A):ZIio{B,A) ^ CHq. 

Proposition 4.9. If A d B are closed smooth manifolds, then H^(B\A) is dualiz- 
ahle. 

Proof. There are two ways to prove this proposition. First, the rational homology 
functor is strong symmetric monoidal, so this follows from [Proposition 2.5| 

We can also show Ht:{B\A) is dualizable directly by describing the coevaluation 
and evaluation. The functor H^{B\A) is supported on isomorphisms and so it is 
enough to construct a coevaluation and evaluation for the chain complexes of vector 
spaces if* (A) and H^,{B,A). These are both finite dimensional, and so they are 
both dualizable with duals as in IScction II □ 

A relative map /: (B,A) {B,A) induces a map 

H^f) : H,(B\A) ^ H^BfA) Zn^(B, A) 

by applying the rational homology functor to /. 

Definition 4.10. The relative global Lefschetz number of /, LB\A{f)i is the trace 
of-ff,(/). 

Lemma 4.11. The image of Lg\^j^{f) under the isomorphism in \Lemma 4.6\ is 

LA(x){f)[x]+ LB(y)/(AnB(y))U)[v\- 

Here Lj^(^^^{f) and is(j^)/(AnB(y))(/) denote the traces of the maps induced by 
/ on H^iAix)) and i/,(B(y)/(A n ^(y))). 

Proof. Using [Proposition 4.9| this proof is similar to the proof of lLemma 4.61 □ 
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The invariant Ls/Aif) is the relative Lefschetz number of [T]. 
Proposition 4.12. In ^(n^'(B, A))), LB\A{f) = iB\A{f)- 



Proof. This proposition follows from |Proposition 2.5| and the observation that the 
rational homology functor is strong symmetric monoidal. □ 



The relative Lefschetz fixed point theorem follows from this proposition and 
[Corollary 4.8l 

Theorem A (Relative Lefschetz fixed point theorem). Let A C B be closed smooth 
manifolds and f: {B^A) — > {B^A) be a relative map. If f has no fixed points then 



Further, if Lg\j^{f) ^ all maps relatively homotopic to / have a fixed point. 



To prove a converse to Theorem A it is necessary to introduce refinements of the 
invariants defined in the previous section. The first of these invariants is the geo- 
metric Reidemeister trace. This is a refinement of the geometric Lefschetz number 



and it will serve as a transition between the global Reidemeister trace in [Section 61 

and the invariant defined in [Section 71 

As for the invariants in the previous section, it is possible to define the geometric 
Reidemeister trace using a generalization of the standard approach of fixed point 
indices and fixed point classes. Also as in the previous section, we do not use 
that approach here. Instead we use duality and trace in bicategories with shadows. 
This perspective gives simple comparisons of different invariants and also unifies the 
descriptions of different forms of the Reidemeister trace with the Lefschetz number. 

Definition 5.1. The relative fundamental category, Ili{B,A), of the pair (B,A) 
has objects the points of B. The morphisms Ili{B, A){x,y) are the homotopy 
classes of paths from x to y in A ii x A and homotopy classes of paths from x to 
y in B if X e B \ A. 

The relative fundamental category is a subcategory of the fundamental groupoid 
of B. In most cases it is not a groupoid. For example, if A and B are both path 
connected, x ^ A, and y ^ B\A then ni(i?, ^)(a;, y) is empty and Iii{B , A){y , x) 
is nonempty. This category is an El-category. This category is similar to the 
equivariant fundamental category, see [32l 1.10.7]. 

For x £ A, let Ax be the universal cover of A based at x. We think of points 
in Ax as homotopy classes of paths in A that start at x. For y & B \ A let By 
be the universal cover of B based at y. Let p: By B he the quotient map and 



Definition 5.2. The relative universal cover of the pair (_B, A) is the functor 



iB|A(/) = 0. 



5. The geometric Reidemeister trace 




Ay=p-^{A) c B, 



B\A: ni{B,A) 



op 



Top, 



defined by 




on objects and by composition of paths on morphisms. 
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Lemma 5.3. If A C B is a cofibration Bx/Ax is Tri{B)-homotopy equivalent to 
Bx^CAx. 

Proof. There is a tti (i3)-equivariant map 

4>: BxUCAx Bx/Ax 

defined by collapsing the cone to the base point. 
Since A <Z B in a, cofibration there is a map 

u: B 

such that u^^(O) — A and a homotopy 

h: B xl ^ B 

such that h{b, Q) — h for all b E B, h{a, t) = a for all a G A and t e / and h{b, 1) E A 
if u{b) < 1. The map 

ip: Bx/Ax ^ Bx^J CAx 

is defined by 

^(7(l),OI[o,2(i-«(7(i)))] 07 if 5 ^ -"(7(1)) < 1 
(M7(l),i)°7,l-2u(7(l))) ifO<ti(7(l)) < i 



^"(7) = 

The map ^ is t^i{B) equivariant. Up to homotopy it is an inverse for 



□ 



Theorem 5.4. If A C B are closed smooth manifolds the relative universal cover 
B\A is dualizable as a module over ni(_B,yl.). 

Proof. The proof of this lemma is very similar to the proof of lLemma 4.31 We will 
define this dual pair by defining a dual pair for each isomorphism class of objects 
in \li{B,A). 

To simplify notation, consider the case where A and B are connected. Let S'^^ 
be the fiberwise one point compactification of the normal bundle of A. This is a 
space over A and has a section given by the points at infinity. Let D[A^) be the 
space {A x.^ S"^^)/ ^ where all points of the form (7, cxd^i-i)) are identified to a 
single point. This is the dual of A^ as a distributor over Tri{A), see [5^ 5.3.3]. 

Let Cb{S'''',S''^) be 

{B X {0}) U {S''^ X /) U (S"'^ X {1}). 

This is the fiberwise cone of the map S"^ S""" over B. Let D{B U CA) be the 
space 

{B XBCsiS'^^S"^))/ ^ 

where all points of the form (7,7(1) x {1}) are identified to a single point. This 
is the composition of the fiberwise spaces {B,p)+ and CsiS"^ , S""^) defined in 
[251 17.1.3]. An argument like that in 5.3.3] for A^ shows this is the dual of 
B U CA as a distributor over ni{B). 
The dual of B\A, denoted D{B\A), is 

D{A+) if x e A 
D{BUCA) ifxeB\A 

The action of the morphisms in Ili(B,A) is by composition. 



DiB\A)ix) = 
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Using the assumption that A and B are connected there are two isomorphism 
classes of objects in Ili{B,A). As in ILemma 4.3| there are four maps that define 
the natural transformation e. Exactly as in that case there are only two that are 
nontrivial. These maps are the evaluation maps for the dual pairs D{A+)) 
and {13/ A, D{BLIC A)). 

Also as in lLcmma 4.3[ B{B\A,Ili{B , A), D{B\A)) is equivalent to 

(i+ A,, (A) V ((B/A) A,,(B) D{B U CA)) . 

The coevaluation map is the composite of the fold map 

gn _^ gn ^ gn 

and the coevaluations for the pairs D(A+)) and {B U CA, D{B U CA)),. 

The required diagrams commute since the coevaluation and evaluation maps are 
defined using coevaluation and evaluation maps from the dual pairs [A^, D{A^)) 
and {B /A, D{BUC A)). □ 

Remark 5.5. We can also give more explicit descriptions of the coevaluation and 
evaluation maps for the pairs D{A+)) and {B/A, D{B U CA)). 
The coevaluation for the pair {A^,D{Aj^)) is the composite 

gn ^ ^ X+ A^,A DiA+) 

of the Pontryagin-Thom map for an embedding of A in S*" with the map v 
(7, 7, v) where 7 is any element of A that ends at the base of v. 

Since A is locally contractible there is a neighborhood U of the diagonal in Ax A 
and a map 

H:V^A^ 

that satisfies H{x, x){t) = x, H{x, y, 0) = x, and H{x, y, 1) = y. The evaluation for 
the pair {A+,D{A+)), 

D{A+) AA+ S"" ATriA+ 

is defined by 

{v,j,S) = {e{v,S{l)),j-'H{S{l),jil))S) 

where e is the evaluation for the dual pair {Aj^, D{A^)). 

The coevaluation and evaluation for the dual pair {B/A, D{B\JCA)) are similar. 

A relative map / : {B,A) {B,A) induces a map 

f,: B\A^ B\AQn({B,A) 

where Il({B, A){x, y) = ^i{B, A){f{y), x)+. The left action is the usual left action. 
The right action is given by applying / and then composition. 

Definition 5.6. The relative geometric Reidemeister trace of/: (B,A) {B,A), 
^'RbiaH), is the trace of the map 

B\A^B\AqTI{{B,A). 

The relative geometric Reidemeister trace is the stable homotopy class of a map 

50^((nf(B,A))) 
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and so it is an element of the 0*'* stable homotopy group of ((n{(i?, A)}). By def- 
inition, the relative geometric Reidemeister trace is an invariant of the relative 
homotopy class of the map. 

Let X be a dualizable space. For a space U and a map 

A: X ^ X AU 

the transfer of an endomorphism f : X ^ X with respect to A is the composite 
S-^ ^ X ADX ^ DX AX "^4-^ DX AX "'-A^ DX AX AU'^'^ S"' A U. 
Let 

AfUA:= {7G A^|/(7(0))=7(l)} 

and 

A^B:= {7 6 5^1/(7(0)) =7(1)}. 
Since A and B are locally contractible and / is taut there are neighborhoods Ua 
of the fixed points of A and Ub of the fixed points oi B \A and maps 

m: Ua^A^^^A 

lb-.Ub-^ A^B 

that take fixed points of / to the constant path at that point. Note that two fixed 
points of / are in the same fixed point class if and only if their images are in the 
same connected component of A-'"'-*^ or B. See [3] or [16] for the definition of 
fixed point classes. 

Let T(7^(/|a) denote the transfer of / with respect to the diagonal map 

A+^A+ AUl/d{TJl) 

and similarly for B. 

Lemma 5.7. If A is a proper subset of B there is an isomorphism 

<(((n((i3,^)))) - 7r^(A/l-A) ©^g(A/i?). 

The image of the relative geometric Reidemeister trace of f under this isomorphism 
is 

iLAUruAf\A)) + {iBUTUnif))- 

Proof. We first define the isomorphism. 

Note that t^q{X) = ZnoiX) for any space X, so it is enough to show 7ro(A^I-*y4) © 
7rQ(A-''B) satisfies the universal property that defines the shadow of Il({B, A). 

The shadow of 'n{{B, A) is defined to be the coequalizer of the maps 

U,MB,A){x,y) X n,{B,A){f{y),x)^^U,niiB,A)if{x),x) . 
The inclusion maps 

{U,eATliiB,A){f{x),x)) n {U,^AlliiB,A){f{x),x)) 7ro(A-^'I^A) © ^;5(A^S) 
define a map 

9: U.,ni{B, A)if{x),x) ^ MA^'^^ A) (Bir'oiA^ B). 

Let a £ Ili{B,A){x,y) and /? G Ili{B, A){f{y),x). U x,y G A then f3a and /(a)/3 
represent the same elements in noiA^^^A). If x,y B\A, j3a and f{a)(3 represent 
the same elements in 7ro(A-^i3). If x and y do not satisfy these conditions, there is 
no condition to check on the paths. So 9 coequalizes. 
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licj): ILx^iiB, A){f{x),x) ^ M is a map that coequalizes the maps above define 
a map 

4>: 7ro(A^''^A) © n^iA^ B) ^ M 

by 0(7) — 4>{P) where (3 is any element of Iii{B , A){f {x) , x) that maps to 7 in 
7ro(A-''l-* A) © 7rQ(A'^_B). This is independent of choices since if a is another hft of 7 
then there are paths and v such that f{fJ.)v is homotopic to /3 and I'fi is homotopic 
to a. Then (j) is unique and 7ro(A-''l-* A) ® 7rQ(A-^_B) is the coequahzer. 

To describe the image of the geometric Reidemeister trace under this isomor- 
phism it is enough to show the trace of 

fU-.A^A 

is ('-a)*(tc/^ and the trace of 

/: B/A ^ B/A 

is {i'B)*{TuB{f))- We will describe the first, the second is similar. 

In [Remark 5.51 we gave explicit descriptions of the coevaluation and evaluation 
for the dual pair Let q: D{A+) A A+ ^ DA A A+ be the quotient 

map. If 771 and ei are the coevaluation and evaluation for the dual pair (^4+, D{A^)) 
and r]2 and €2 are the coevaluation and evaluation for the dual pair DA^) then 
the explicit descriptions of 771 and ei show 

S"^^A+ A,,aD{A+) 



A+ADA 

and 

iD{A+) A i+ A n({A))) 5" A((nf 

DA A A+ "^""^ - DAaA+A UA/dlJA 5"" A IJA/dUA 

commute. 

Together these diagrams show 

(ei Aid)(/Aid)777i = (idAiA)(e2 Aid)(idAA)g(/ Aid)7?7i 
= (id AiA) (£2 A id) (id A A) (/ A id)7g77i 
= (idAiA)(e2 Aid)(idAA)(/ Aid)7?72 

The first composite is the trace of /|^. The last composite is (tA)*(''';7^ (/U))- D 

For a fixed point class /3 of / : S ^ S let z^*^' be the index of the fixed points 
associated to /? that are contained in B\A. For a fixed point class a oi f\A'- A ^ A, 
let ia be the index of the fixed points associated to a. Since the map / is taut, Zq, 
is the fixed point index of the fixed points in A with respect to either f\A or /. 

The following corollary is a consequence of lLemma 5.71 and is the generalization 
of ILemma 4.61 This corollary identifies the relative geometric Reidemeister trace 
with the generalization of the classical description of the Reidemeister trace. 
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Corollary 5.8. Under the isomorphism in \Lemma 5. T\ 

The following two examples were described in [24j . In that paper the generalized 
Lefschetz number and one form of the relative Nielsen number are computed. 

Example 5.9. [H 5.1] Let B ^ x and A ^ x S\ Let f : B ^ B he 

/(re^^e'*) = (/l(r)e-'^e3»*) 

where /i : [0,1] [0,1] is a continuous function such that /i(0) = 0, /i(l) = 1 
and /i has no other fixed points. Then / is a relative map with six fixed points. 
There are four fixed points in A. These fixed points all represent different fixed 
point classes and all have index —1. The two fixed points outside of A represent 
different fixed point classes in B and also have index —1. 

Since A is a torus, t^i{A) = (a, b\abab = 1). The relation imposed on the shadow 
implies {tti{A)'^} consists of 4 elements, 

1, a, 5, ah. 

For B, TTi{b) = (b) and ((7ri(i?)'^)) consists of 2 elements, 

1,6. 

Then 

'^'"RbiaU) = + aA + bA + abA + Ib+ bs)- 

Example 5.10. [H 5.2] Let B = 5^ x 5^ and A = 1 x 5^. Let / : B ^ 5 be 

There are three fixed points of / in A and three additional fixed points of / in 
B\A. 

The three fixed points of / in A represent each of the three possible fixed point 
classes. These fixed points all have index 1. The three fixed points in B that are 
not in A also represent three distinct fixed point classes, but these are only three 
of the six possible fixed point classes. These fixed points also have index 1. 

Let 7ri(B) = {a,b\abab = 1). Then 7ri(A) = (a). The set ((7ri(B)'^)) consists of 

1, a, a^, 6, ab, a^b. 

The set {{ni{A)'>')) consists of 

1, a, a^. 

Then 

^^RbIaU) ^lA + aA+a\ + bB + {ab)B + {aH)B. 

The relative Nielsen number. One of the expectations for the Reidemeister 
trace is that it can detect when a map has no fixed points but it does not have to 
provide a lower bound for the number of fixed points. This is very different from 
the Nielsen number. The goal of the Nielsen number is to provide a lower bound. 

In the classical case, the Nielsen number is the number of nonzero coefficients 
in the Reidemeister trace. This implies the Nielsen number is zero if and only if 
the Reidemeister trace is zero. For more general situations the connection between 
nonzero coefficients of the Reidemeister trace and the Nielsen number does not hold. 
It remains true that the Nielsen number is zero if and only if the Reidemeister trace 
is zero. 
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The inclusion of A into B induces a map ni{A) 7ri(_B) and also induces a map 
$ from the fixed point classes of A to the fixed point classes of S. A fixed point 
class of / or /l^ is essential if its coefficient in the classical Reidemeister trace is 
nonzero. Let 

^(/,/U) 

be the number of essential fixed point classes of B that are in the image of an 
essential class of A. Let N{f) be the classical Nielsen number of / and N{f\A) be 
the classical Nielsen number of 

Definition 5.11. ^35i 2.5] The relative Nielsen number, N{f;B,A), is 

N{f\A) + {N{f)-N{fJ\A)). 

Lemma 5.12. The relative Nielsen number of f is zero if and only if the relative 
geometric Reidemeister trace of f is zero. 

Proof. If the relative geometric Reidemeister trace of / is zero [Corollary 5. 8| implies 
(X)*/?"^'/^) + (X]*qQ;) is zero. Since 1I(Il{{B, A)} is a free group generated by the 
a's and (3's each i^'^'- and ia are zero. Since the i^s are zero, N{f\A) and N{f, 
are zero. Since each of the ia's are zero ip = i^*^' = for every /?. This implies 
N{f) is also zero. 

By definition A^(/|^), N{f ), and N{f, /|^) are all greater than or equal to zero 
and N(f,f\A) < N{f). If the relative Nielsen number of / is zero 7V(/|^) = 
and iV(/) = NifJU). Since iV(/U) - 0, iV(/,/U) = and so iV(/) = 0. Since 
^(/U) — all of the i^s are zero and ijj'^' = ip. Since N{f) = 0, ip — for all 
p. □ 

The relative Nielsen numbers for the maps in the examples above were computed 
in |24| . The relative Nielsen number for [Example 5.9| is 4. This is not the number of 
non zero coefficients in the relative Reidemeister trace. The relative Nielsen number 
for [Example 5.10| is 6. This does happen to be the number of nonzero coefficients 
in the relative Reidemeister trace. These numbers coincide because N{f,f\A) is 
zero for this example. 

Other references for relative Nielsen theory include [15l [281 EHl EH [34] . These 
invariants are also related to the Nielsen numbers for stratified spaces defined in 

m- 

6. The global Reidemeister trace 

In this section we define the relative global Reidemeister trace. This invariant is 
a generalization of the relative global Lefschetz number and can be identified with 
the relative geometric Reidemeister trace. The relative global Reidemeister trace a 
relative generalization of the invariant defined in [13j . It is related to the invariants 
defined in [24l[36], but it is not the same as either of these invariants. 

Let 'ZIli{B,A) be the category with the same objects as Ili{B,A). The mor- 
phism set 

ZIli{B,A){x,y) 

is the free abelian group on the set Ili{B, A)(x, y). 
There is a functor 

C,{B\A): Zni(B,A)°P ^ ChQ 
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defined by C^,{B\A){x) = C^,{B\A{x); Q) where the second C* indicates the cellular 
chain complex. The action of the morphisms of Ili{B,A) is induced from the 
action on B\A. This functor is defined in the same way that the functor H,,{B\A) 
is defined from the functor B\A except we replace the rational homology functor 
with the rational chain complex functor. 

Proposition 6.1. If A C B are closed smooth manifolds the Uli{B, A) -module 
C^,{B\A) is dualizahle. 

Proof. Like [Proposition 4.9| there are two possible proofs of this theorem. 

The rational cellular chain complex functor induces a functor on bicategories, 
and for A C B closed smooth manifolds. ITheorcm 5.4l shows that B\A is dualizable. 
[Proposition 2.5| then implies that C^,{B\A) is dualizable. 

There is a second approach using iLemma 3.7l li x ^ A, C^,{B\A){x) = C^,{Ax) as 
a module over Tri{A, x). This is a finitely generated free module and so is dualizable 
with dual 

Homz^^(^_^)(C*(i^),Z7ri(^,x)). 

If X E B \ A, C^,{B\A){x) — C^,{Bx/ Ax) as a module over 7ri{B,x). This is also a 
finitely generated free module and so is dualizable with dual 

Homz^^ (^B,x){C*{Bx/Ax), Ztti {B,x)). 

Since C^{B\A) is supported on isomorphisms . iLcmma 3 . 7l implies C^,{B\A) is dual- 
izable. □ 

A map /: {B, A) {B, A) induces a map 

C4B\A) ^ C4B\A)QZn{{B,A). 

Since C^:{B\A) is dualizable, the trace of /* is defined. 

Definition 6.2. The relative global Reidemeister trace of /: {B,A) — > {B,A), 
^'RB\A{f)i is the trace of 

C4B\A) -> C4B\A)QZn{{B,A). 
The relative global Reidemeister trace of / is a map 

Z^m{iB,A)}. 
Lemma 6.3. If A is a proper subset of B then 

((n{(i?,A)))-((n{(i3)))n((n{(A))). 

The image of ^^RB\A{f ) under this isomorphism is 

E ''RUW.))[X]+ J2 "mBiy)/iBiy)nA))[y]- 

Here ^?R(/U(£c)) denotes the usual global Reidemeister trace of f\A(x) a-s defined 
by Husseini in 13 . The invariant ^ii{f\B{y) U Cf\B{y)nA) is the trace of 

/. : C4B{y)/iBiy) n A)) ^ n A)) n({B,y) 

as a module over T:i{B,y). 
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Proof. To simplify notation, consider the case where A and B are connected. The 
proof is similar if A and B are not connected. 

The shadow is defined to be the coequalizer of the maps 

U,,yIl,{B,A){x,y) X ni{B,A){fiy),x)^^UMB,A){f{x),x) . 

Instead of indexing these coproducts over all objects in Ili{B,A) we can index 
over representatives of each isomorphism class of objects in Ili{B,A). This gives 
four terms in the first coproduct. The two cross terms are both empty and so this 
coequalizer splits into the coequalizer that defines ((n{(i?))) and the coequalizer that 
defines ((n{( A)}). 

For the second statement, note that this isomorphism is compatible with the 
description of the dual pair. Then the trace is the pair of classical traces. □ 

This description of the relative global Reidemeister trace shows that the second 
coordinate is the the relative Reidemeister trace of j36]. This also shows that 
this invariant is related to, but not the same as, the generalized Lefschetz number 
defined in [24] . 

Proposition 6.4. If A C B are dosed smooth manifolds and f: {B,A) — > {B,A) 
is a relative map then 

''RB\A{f) = 'KlAif)- 

Proof. Since both ®^i?s|A(/) and ^5?b|a(/) are defined as traces and the rational 
cellular chain complex functor is strong symmetric monoidal this proposition follows 
from |Proposition 2.5| □ 

7. A CONVERSE TO THE RELATIVE LeFSCHETZ FIXED POINT THEOREM 

There are several proofs of the converse to the relative Lefschetz fixed point 
theorem. Some, like [28l [29] [34] , are generalizations of the simplicial arguments 
used in the standard proof of the converse to the classical Lefschetz fixed point 
theorem, see [3]. 

In this section and the next section we describe a proof of the converse to the 
relative Lefschetz fixed point theorem that follows the outline of [19l [18] . This 
approach is not simplicial and it easily generalizes. For example, see |18j for the 
equivariant generalization and [26 for the fiberwise generalization. 

The approach of [T9] is based on invariants that detect sections of fibrations. 
In the next section we prove relative generalizations of the results in |19j . In this 
section we apply those results to relative fixed point invariants. 

The main result of this section is: 

Theorem B (The converse to the Relative Lefschetz fixed point theorem) . Suppose 
A C B are closed smooth manifolds of dimension at least 3 and the codimension of 
A in B is at least 2. The relative global Reidemeister trace of a map 

f: {B,A)^{B,A) 

is zero if and only if f is relatively homotopic to a map with no fixed points. 

The first step in the proof of Theorem B is to describe relative maps without 
fixed points in terms of relative sections. 
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Lemma 7.1. Let A C B be closed smooth manifolds. Relative homotopies of a map 
f : {B, A) {B, A) to a relative map with no fixed points correspond to liftings that 
make the diagram below commute up to relative homotopy 

{B X B\A,Ax A\A) 



iB,A) 



{B X B,Ax A). 



The function Fy is the graph of / and T f(m) — (m, f{m)). 

Proof. If / is relatively homotopic to a fixed point free map g via a relative homo- 
topy -ff, then Th is a relative homotopy from T f to Tg. 
For the converse, suppose there is a relative map 

k: {B, A) ^ {B X B \ A, A X A \ A) 

and a relative homotopy K from k to F/. 

If A is a smooth manifold the first coordinate projection 

proji : Ax A\A^ A 

is a fiber bundle and there is a lift Ja in the diagram 

k 



A- 



Ja 



Ax A\A 



■■■ projiA' 

A X I ^ A. 

Since A C -B is a cofibration and projj^ : B x B\A ^ B is a. fibration the diagram 

kUJA 



BuAxI- 

io 

B X I — 



■B X B\A 



,..--y 



projl 



projioA' 



B 



has a lift J extending the lift J a above, see fSl] Theorem 4]. Note that projj^ o 
J(— , 1) = id. Let g = proj2 J(— , 1). This map has no fixed points. 



The homotopies K and J define a relative homotopy from F f to F„ 



□ 



Given a map / : y ^ y, let r{f) : N{f) Y denote a Hurewicz fibration such 
that 




commutes and V — > N{f) is an equivalence. 

Lemma 7.2. Let X cY, p: My Y be a space over Y and Mx C p~^{X). 
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Liftings up to relative homotopy in the diagram 

{My,Mx) 

{B,A)'^^ {Y,X) 

correspond to relative sections of the pair of fibrations 

{9*N{fY),g*N{fx))^{B,A). 

If p: E' — > _B is a Hurewicz fibration the unreduced fiberwise suspension of p is 
the double mapping cylinder 

SbE := B X {0} UpE X [0, l]UpB x {!}. 

The map p: E B defines a fibration 

SbE B. 

There are two sections of this fibration, ai,a2'. B ^ SbE, defined by the inclusions 
of _B X {0} and B x {I}. If S*^ := i? 11 B, these sections define an element of 

[S%, SbE]b- 

Let iB- BxB\A^BxBhe the inclusion. Then the pair of fibrations 
{rj4N{zB)),Tj,{N{iA)))^{B,A) 
determine an element in 

[S"B,SBrf,{N{iB))]B © [S%SATf4N{tA))]A- 

This element will be denoted ^^RB\A{f)- 

Proposition 7.3. Let A G B be closed smooth manifolds of dimension at least 3 
such that the codimension of A in B is at least 2. A continuous map 

f: {B,A)^{B,A) 

is relatively homotopic to a map with no fixed points if and only if ^^RB\A{f) = 0. 

The proof of this proposition, except for one key step proved in the next section, 
follows the preliminary lemma below. 

Lemma 7.4. [19l 6.1, 6.2] Let M be a manifold of dimension n and i : MxM\A^ 
M X M be the inclusion. Then T f^(N{i)) —>■ M is (n — l)-connected. 

Proof of \Proposition 7~3[ ILemma 7.11 and ILcmma 7.21 convert the question of find- 
ing a lift of a relative map /: {B, A) {B, A) to the question of finding a section 
of the fibration 

(Tf^iNitB)), (r^|,,(7V(*^))) (B, A). 
If the dimension of A is ua and the dimension of B is ub then lLemma 7.4l implies 
that Tf^,{N{iB)) B is {hb — l)-connected and rf^^^,{N{iA)) A is {ua — 1)- 
connected. If ua and hb are at least 3 and — ua is at least 2, [Proposition 8"!6| 
implies that 

iTf4N{tB)),rf\,4N{zA))) ^ {B,A) 
has a relative section if and only if =0. □ 
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The hypotheses in this proposition are not the standard hypotheses used in the 
converse to the relative Lefschetz fixed point theorem. The standard condition is 
that 

m{B\A)^7ri{B) 

is surjective. The codimension condition impUes this condition. We use a codimen- 
sion condition since it is compatible with the approach of the proof. I don't know 
if the surjectivity condition can be used in this approach. 

To complete the proof of Theorem B we need to compare ^'^RbiaU) the 
relative geometric Reidemeister trace. 

Proposition 7.5. Let A C B be closed smooth manifolds and /: {B,A) {B,A) 
be a relative map. Then 

"""^RbiaU) = if and only if K\A{f) = 0. 

We recall a lemma from |18] . 

Lemma 7.6. [18[ 7.1] [26[ 8.3.1] Let M be a closed smooth manifold with normal 
bundle vm- Then there is a weak equivalence 

S""' QTf^SMxMNiiM) ^ 5" A A^M. 

Proof of \Proposition Suppose X and Y are ex-spaces over B, the projection 
maps are fibrations and the sections are fiberwise cofibration. Let {X, Y}b denote 
the fiberwise stable homotopy classes of maps from X to Y. 

If A and B are both closed smooth manifolds of dimension at least three, then 
the dimension assumption, ILemma 7.41 and the fiberwise Freudenthal suspension 
theorem in [ij, 4.2] imply that the maps 

[s%SArfu4N{^A))]A - {slSArMN{tA))}A 

[S%,SBTf4NizB))]B ^ {S%,SBTf4NizB))}B 

are isomorphisms. Costenoble-Waner duality 23, 18.5.5, 18.6.3] and ILemma 7.61 
imply there are isomorphisms 

{S°A,SArf\^^{N{lA))}A = {S^,S-'^ QSATf^NitA))} 

^ {S",S'' AAf\^A+}. 

and 

{S%,SBrf,{N{tB))}B - {S'\S''^ QSBTf^iNitB))} 

^ {S'\S'' AAfB+}. 
Let Ua be a neighborhood of the fixed points oi f\A such that there is a map 

la:Ua^ A^l-^yl 

that takes fixed points to the constant path at that point. In [521 6.3.2] it is shown 
that the image of in ^g(A/I^A+) is LA{T{f\uJ). 

Let Ub be a neighborhood of the fixed points of / in i? \ A such that there is a 
map 

lb-Ub^ A^B 
which takes fixed points to constant paths. 
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The image of ^^b|a(/) in '''o(^'^^+) is the composite of the transfer of / with 
respect to the diagonal map 



B+^B+A [Ub n UA)/d{UB n Ua) 

with the map 

t := M n IB : C/a n C/s A-'' B. 
Since the transfer is additive, [7], the image of ^^Rb\a{I) in ^o(^''^-^+) i^ 
i{TUunuAf)) - i{TUsU)+ruAf\A)) = ^{wM)) + ^{ruAfU)). 
Then ^^i?B|^(/)iszero£andonlyif m(t(/|[/J) and t(ry, (/)) + .(r^^ (/U)) 
are both zero. Using [Lemma 5. 7l these elements are zero if and only if ^Rb\a{I) is 
zero. □ 

Proof of Theorem B. [Proposition 7.3| implies that / is relatively homotopic to a 
fixed point free map if and only if ^^'RB\A{f) ~ 0. [Proposition 7.5| implies 
'^^RB\A{f) = if and only if ^''RB\A{f) = 0. [Proposition 6:4[ implies <'^RB\A{f) = 

''RB\A{f)- □ 



Remark 7.7. [Proposition 7.3 and the proof of [Proposition 7.5[ show if dim(A) > 3 
and dim(B) > dim (A) + 2 "^^RgiAif) 

is zero if and only if the two nonrelative 

invariants for A and B are zero. 

Using these two invariants to define a relative invariant would be analogous to 
defining the relative invariants in the previous sections as the pair of classical in- 
variants for the spaces A and B. This alternate definition would satisfy the require- 
ments of the introduction for a fixed point invariant. However, there are several 
reasons why the corresponding definition in the equivariant case is not acceptable. 
The definitions in the previous sections were chosen because they are consistent 
with the choices in [25] . 

8. Relative sections 

In this section we generalize the result from 19 on sections of fibrations to 
relative fibrations. 

If the dimension of B is 2n and the fibration p: E ^ B is n+ 1-connected, it is 
shown in [19] that the two sections 

(Ti, (72 : B ^ SbE 

are homotopic over B if and only there is a section of p. We can generalize this 
result to relative sections. 

li A<Z B let Ea be a subspace of Eb such that the image of p restricted to Ea 
is contained in A. Let Sa.bEa be 

B X {0}UEax lUAx {1}. 

Let [{Sg, A n B), {SbEb, Sa,bEa)]b be the relative fiberwise homotopy classes of 
maps from (S*^, A U B) to {SbEb, Sa.bEa)- 

Definition 8.1. Let A C B, p: Eb B he a, fibration, and Ea C such 
that Ea — > A is a fibration. The relative homotopy Euler class 

X(^[{S%,A\1B),{SbEb,Sa.bEa)]b 
is fJi U (72 : S"^ ^ SbEb- 



28 



KATE PONTO 



Proposition 8.2. // {Eb,Ea) (B^A) admits a relative section ^ then x is 
trivial. 

Conversely, assume p: Ea A is (to + \)-connected, A is a 2m- dimensional 
CW-complex, p: Eb B is [n+V)- connected and {B, A) is a relative 2n- dimensional 
CW-complex. If x is trivial then p has a relative section. 

Before we prove this proposition we recall a preliminary lemma. 

B be a (j + I) - connected fibration and P be the 



Lemma 8.3. [HI 3.1] Letp: E 
homotopy pullback 



B 



B 



SbE. 



A fiherwise homotopy from ai to defines a 2 j -equivalence q: E ^ P. 
Proof of \Proposition 87^ If there is a relative section <j then the homotopy 
H: iS%,AUB) X I ^ {SbEb,Sa^bEa) 

defined by H{b,t) ~ (f(6),t) shows x is trivial. 

If X is trivial there is a relative fiberwise homotopy 

K: {S%,AUB) X I ^ {SbEb,Sa^bEa) 

from (72 to (Ti. The restriction of K to Sa defines a homotopy between cri^ : A — > 
SaEa and a2\A. ILemma 8.3[ Whitehead's theorem, and the homotopy K\go imply 

qA*: [A,Ea] ^ [A, Pa] 

is a bijection. The space Pa is as in ILemma 8.31 

The restriction K\go^ induces a map Ha'. A ^ Pa such that phA = id. Since qA* 
is a bijection there is a map kA'. A Ea and a homotopy Ja from qA^A to Ha. 
Then pkA = piqAkA) — phA — id^ via the homotopy p(Ja). The diagram 



A- 



kA 



■Ea 



La 



Ax I 



p{Ja) 



A 



has a lift La, and p(Lyi(a, 1)) = a. Then , 1) is a section of p^^(A) A that 

is contained in Ea- 

The homotopy K defines a map hs-B^ Pb extending the map hA . The space 
Pb is as in ILemma 8.31 The homotopy extension and lifting property implies the 
dotted maps in the following diagram can be filled in 



A' 



B 



Ax P 




Pb 




Jb 



■B X I- 




Eb 



kB 



B 
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defining maps ks and Jb extending kA and Ja- 

Since the pair {B, A) has the relative homotopy lifting property there is a lift 
Lb in the diagram 

BU{AxI) JlE}±±^ Eb . 



B X I- 



p(Jb) 



B 



Evaluating at 1, 1)) — pJB{b,l) = phB{b) = b. Since ^^(a,!) G Ea for 

a ^ A, Lb{—, 1) is the required section. □ 



ILemma 7.1[ ILemma 7.2[ and [Proposition 8.2| imply x is a complete obstruction 
to determining if a relative fibration has a section. In the examples we are interested 
in, it is easier to work with invariants defined from x than with x itself. Under 
some additional hypotheses, these associated invariants are zero if and only if x is 
zero. 

If A C B, define 

Cb{B, A) -.^ B X {OjuAx [0, l]UB X {!}. 

This is an ex-space over B with section given by the inclusion of B into Cb{B, A) 
as B X {0}. 

In the diagram below the vertical maps are induced by cofiber sequences, [U 
II. 2. 4] and so the columns are exact. The horizontal maps are forgetful maps. The 
diagram commutes. 

XB,A e [{CBiB, A), SbA), (SbEb, SaEa)]b ^ [CBiB, A),SbEb]b b Xb,a 



XA 



[AUB,Sa,bEa]b 



[S%,SbEb]b b Xb 



[AnB,SBEB]B B XA 



The elements XA, Xb, and XA are the images of x- The element xb,a is defined if 
XA = 0. Then xb,a is the preimage of x- The element xb.a is defined if XA ~ 0. 
Then Xb,a is the preimage of xb- 

Lemma 8.4. If xb.a — then xb.a — 0. 

Proof. Suppose Xb,a — 0. Then there is a fiberwise homotopy 

L: Cb{B,A)xI ^ SbEb 

such that 

L(6,l,0) = a2{b) 
L(6,l,l) = ai{b) 
L{b,0,t) = ai{b) 
L(a,s,0) xs,A(a, s) e S'a-Ba 

L(a, s, 1) = cri(a) 
for all a E A, b E B, and s,t E I. 
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Let J := ({0} X J) U (/ X {1}) U ({1} x /). Define a map 

L: B xJ-^SbEb 



by 



L{b,0,t) 

L{b, .s, 1) 



ai(6) 



The diagram 



(B X J) (A X / X /) ^^^'^'''y SBi;^ 



if 



B xl X I- 



-^B 



commutes and there is a lift K since SbEb B is a fibration. 
Then Kq := K{-, -,0): B x I ^ SbEb satisfies 

Ko{b,0) = if(6,0,0) = L(6,0,0)=cri(6) 
Koib,l) = i^(6,l,0) = L(6, 1,0) =(72(6) 
Ko{a,s) = K{a,s,0) = L{a,s,0) £ SaEa 



Define a map 
by 



K: Cb{B,A)xI ^ SbEb 

K{b,l,t) = Ko{b,l-t) 

Kib,0,t) = ai(6) 
k{a,s,t) = Ko{a,s{l-t)) 

k shows xb,a is trivial in [{Cb{B,A),SbA), {SbEb, SaEa)]b- 



□ 



Lemma 8.5. // the map Eb B is a (dim(A) + 1)- equivalence then p is injective. 

Proof. In this proof let i denote the inclusion oi A in B. 

Let Eb{AUB) := {{A x I)UB)/ ~ where (a,0) ~ i(a) ~ (a,l). Then p is part 
of a long exact sequence 

n B),SbEb]b ^ [Cb{B, A), SbEb]b ^ [S%, SbEb]b ^[AUB, SbEb]b. 
To show that p is injective it is enough to show 

[1:b{AUB),SbEb]b 

is trivial. 
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Let a be an element of [Sb(A 11 i?), SbEb\b- Then a defines a map 5*^x^4- 
SbEb also denoted a. This map satisfies pa(t,a) ~ i{a). Consider the diagram 



S"! X A X / ■ 



■S^ xA 



B 




ioproj 



H 



SbEe 







xA- 



X Ax I- 



X A 



Since SbEb B is a (dim(j4) + 2)-equivalence, the homotopy extension and lifting 
property implies there are maps P and H that make the diagram commute. 
The diagram 

xA)U,S^xAxI \ SbEb 

H 



B 



X Ax I — 

commutes. Since SbEb B is a. fibration there is a lift K that makes the diagram 
commute. Then 

Ko :=i^(-,-,0): D^xA^SbEb 



satisfies 
and 

ifweS\ Then 



pKq{v, a) = H{v, a, 0) — i{a) 
Ko{w, a) — a(w, a) 



KoUid: {{D^ X A)UB)/ > SbEb 

defines a map that shows a is trivial. 



□ 



The following proposition is a consequence of |Proposition 8.2\ ILemma 8.41 and 
ILemma 8.51 

Proposition 8.6. Ifp: Ea —>■ A is (m + 1)- connected, A is a 2m- dimensional CW- 
complex, p: Eb B is {2m + 1)- connected and {B,A) is a relative 'im- dimensional 
CW-complex {Eb, Ea) {B, A) admits a relative section if and only if XA and xb 
are both zero. 



9. Other descriptions of in special cases 



These are the proofs omitted from 1 Sect ion 31 Let ^ be an El-category enriched 
in the category of abelian groups. 



Lemma 9.1 (jLemma 3.6p . If X : si Ch/j and '¥ : — ^ Chp are supported 
on isomorphisms 

ceB(i/) 
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Proof. We will show that ©^(c) ®.c/(c.c) ^ (c) satisfies the universal property that 
defines J" ^. 

By definition of B{£/), for any object a in £/ there is exactly one object 

c e B{£^) 

such that there is an isomorphism / : a — > c in x/. Define a map 

0a ■■ ^(a) ®Z ^{d) ^ ^(c) ^r(c) 

as the composite of 
with the quotient map 

^(c) ®z ^(c) ^ Jr(c) ®.^(,,,) ^(c). 

If g is another isomorphism in from a to c, then {,!%' {f){A),'3^{f^^){B)) is 
identified with (jr((7)(>l), ?V{g~^){B)) and the map is well defined. Let 

oeob(j2/) ceB(.s!/) 

be the sum of the maps 9a- 

If (A, /, B) £ ^{a) ®z jz/ (a, 6) 0^ ^ (&) the images of this element in 

®aeohi^/)^{a) «>z^(a) 

are {A,'3^{f){B)) and ( B). The images of these elements are identified 
under 9. 
Let 

a^ohsi' 

be a map that coequalizes the two maps from ®a,f)eob.c/>^(a) £^{a,b) ?^{b) 
to ©a6ob.<i/>^(a) ®z ^^(a). Define a map 

V-: ^(c) ®.^(,^,) ^r(c) ^ M 

by choosing lifts of elements in jr(c) (X)^(c,c) ^^(c) to elements of 

S-(c) ®z ^^(c). 

Since coequalizes, the choices do not matter and ip is unique. □ 

Lemma 9.2 (jLemma 3.7|) . Let ^ and '3^ satisfy the conditions of lLemma 3.6[ If 
^ {c) is dualizable as a £/{c, c) -module with dual '3^{c) for each c G B{s^) then ^ 
is dualizable with dual '3^ . 

Proof. If ^(c) is dualizable as an ^(c, c)-module with dual ^^{c) then there is a 
map of chain complexes of abelian groups 

Vc-.z^ jr(c) 0^(c) 

and a map of chain complexes of £/{c, c)-bimodules 

ec: ^(c)0S'(c) ^^/(c,c) 

for each c G B{£/). 
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Let ry : Z ^ j?^' be the composite 

where A : Z — > ®s(^)Z is the map that takes 1 to (1,1,...,!). 

Let a and b be isomorphic objects of s/. Let c be an object of B{s^) that 
is isomorphic to a and let h be an isomorphism in from a to c and be an 
isomorphism from b to c. Then ea,6 is the composite 



^(c) 0z jr(c) 



c, c 



r(3-i)(»,r(/i) 



If a and b are not isomorphic in €a,b is zero. Since c is unique and the maps 
Cc are maps of si/ (c, c)- bimodules, e is a natural transformation. This also implies 
that e is independent of the choice of g and h. 

Let r]c{l) = ^^ec,i (8 /c,i for each c € B{^). If a; € ^(a) the value of the 



composite 
applied to x is 



jr(a) S Z O ^(a) "-^^ ^ ^ O jr(a) JT ^(-, a) ^ ^{a) 



J2 J2^Wo,i'^)K^c,i). 



The only nonzero terms in this sum are those where there is an isomorphism h from 
X to c. By definition, e{fc,i,x) = h~^ec{fc,i, ■S^{h){x)) and 

^ jr(e(/c,i,ar))(ec,0 = ^{h-^)Y,^{eo{fc,i, 3^{h){x))){e,,i) 

= ^{h-'^)d{h){x) 
= X. 

The other diagram is similar. □ 
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